Simulation, virtual reality and augmented reality is a growing industry and stands to supplement, and in some cases, replace conventional training atmospheres. Using unique VR and AR technologies, real-time interaction is imperative in order for a user to become mentally and emotionally immersed in the computer-generated virtual environment. Thus the primary goal when creating any simulation, virtual reality or augmented reality environment is to ensure a seamless simulated environment without creating distractions that would deviate the attention of the user away from the environment. Two common problems include lag time and wire tethering of users.
One of the main goals in virtual reality is to create a completely wireless system such that the user may become immersed in the virtual or augmented reality. Realistic virtual reality systems for training or entertainment require the system to be customized for safe use by a human being in motion As a result, have a user being tethered with electrical power or data carrying wires to any sort of system is undesirable. In addition, the slightest lag time or buffering of information into such environments can ruin or interrupt the environment, thus making such training ineffective. In addition, power and signal wires connected to head mounted displays (HMDs) interfere with the VR and AR immersive experience. However to date present attempts at a wireless VR and AR immersive experience solution increase lag time significantly making an unusable virtual or augmented reality.
Use of wireless HMDs is technically challenging due to issues of weight, battery power, display quality, and image latency. Video displays like those used in cell phones are lightweight, have excellent display quality, and require low power consistent with battery-powered use. However, a critical additional requirement for acceptable use by humans is to minimize the lag between the motion of the user's head and the corresponding updating of the visual displays. This lag time is very distracting and can induce nausea, preventing a user from using the device to become immersed in a virtual or augmented simulation.
Video display equipment, including computer graphics cards and video display screens, must conform to video standards set by electronics industry organizations such as the Institute of Electrical and Electronic Engineers (IEEE). In addition, the Federal Communications Commission defines radio frequencies and other details related to radio transmissions including transmission of wireless video signals. The WirelessHD Consortium is a group of independent companies that work together to promote technological advancement and adoption of wireless video equipment in compliance with established video standards and FCC regulations. At this time, the member of the WirelessHD Consortium that designs and manufactures WirelessHD-related integrated circuits is the company Silicon Images (a subsidiary of Lattice Semiconductor). Existing attempts to create a wireless virtual technology used wireless video standards evolved from market demand to connect statically placed home/office television equipment, such as connecting a DVD player or set-top box to a wall-mounted television. As a result, commercial off-the-shelf wireless video equipment conforms to legal (such as the FCC) and operational standards that were developed with home/office televisions and monitors in mind, without thought to application to virtual reality. Notwithstanding, existing virtual reality components make use of many of this type of equipment, thus there remains an unmet need for virtual reality grade wireless video equipment, or a system or method for augmenting video signals from existing wireless video equipment to make the video signals better adaptable to virtual reality use. In addition, wireless (untethered) virtual reality and augmented reality imposes additional demands on video, audio, and data. VR/AR applications use head or helmet mounted displays (HMDs), which despite their name including “display” also include headphones and microphones for audio and controls that need to be remotely controlled. These off-the shelf systems are poor for VR systems in several ways because a) they have a limited visual resolution not exceeding 1920×1080, b) have a limited refresh-rate set at 60 Hz where 90 Hz is preferred for VR, c) limits transmissions to a single user; d) only single video streams can be processed at one by a transmitter/receiver pair; e) transmission beam only dynamically altered to provide a low-strength reflected signal (see FIG. 1A) relative to a receivers position; f) the WirelessHD (WiHD) Receiver is actually a Receiver/Transmitter combination, thus the “Receiver” radiates millimeter-wave energy that's harmful to humans above certain limits, and FCC certification requires placement of the Receiver at least 20 cm (˜8 inches) away from the user; g) audio is transmitted without audio-related control ability; h) there is no capability to remotely control or pass instructions to the HMD; and i) there is no capability to pass other (non-command) data to the HMD. Thus there remains an unmet need for a virtual reality video, audio and data communications equipment that overcomes these shortcomings, which allows for the placement of receivers directly on a user, which provides a line of sight transmission to a moving receiver worn by a virtual reality user, and which provides a way to pass commands and data to the equipment worn by the user.
In addition to the above shortcomings, conventional over the air video transmitters and receivers have a high power consumption, and thus are not very good candidates to be battery powered. As a result in order to use them either a large amount of weight in the form of batteries needs to be carried by the user, the simulated environment is limited to short periods of time reflecting the particular battery life, or requires that a user be tethered to a power source. A key point, critical to the subject of this patent application, is that cell phone displays are natively Portrait mode devices but WirelessHD transmitters, for example those currently produced by Silicon Images, can only transmit a 1080p high resolution image in Landscape mode. To use a Portrait display device, such as a cell phone display screen, with a WirelessHD receiver that must operate in Landscape mode, the received image must be rotated ninety degrees. This required image rotation can be done by buffering an entire frame of the image, then rotating the entire image, but that approach induces one full frame of latency. At 60 frames per second, each frame is delayed by 16.7 milliseconds. Such a long lag time is widely considered unacceptable for real time immersive virtual reality and augmented reality systems because that long lag is perceptible, distracting, and sometimes sickening to humans in immersive environments.
Thus each shortcoming described above provides major disruptions to a simulated, virtual or augmented reality, and thus there remains an unmet need for wireless video processing equipment that has a low power consumption while providing a reliable and seamless virtual or simulated environment. In addition, there further remains an unmet need to reduce the latency caused by image rotation in such video systems.